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Conclusions 


A short survey was sent to Alabama high schools with 
greater-than-expected ACT® WorkKeys® performance with 
the goal of identifying effective practices for promoting 
career readiness. Six of the 10 schools contacted 
responded to the survey. All respondents integrate career 
readiness into their regular curricula, whether through 
co-op programs, dual-credit initiatives, or certification 
programs. All responding schools use WorkKeys-specific 
test preparation materials and set aside class time for 
students to engage with those materials. Many of the 
responding schools have relationships with businesses in 
their communities and communicate hiring requirements 
from these businesses to their students. These 
relationships are additionally fostered through career fairs. 


So What? 


This study highlights the importance of clear 
communication in fostering career readiness. Students 
exhibit higher motivation on achievement tests when they 
understand the value of an assessment and are able 

to familiarize themselves with the exam prior to testing. 
Increased communication regarding the work requirements 
of local employers can further contribute to career 
readiness. 


Now What? 


While several of the interventions described by survey 
respondents would require significant time and effort (e.g., 
integrating career programs into regular curricula), many 
of the practices outlined in this report would be, in terms 
of time and effort, easy to implement. Communicating 

the work requirements of local employers could be done 
directly through networking with local businesses who hire 
high school graduates or indirectly through organizing a 
career fair. Supplying links to practice test content and 
scheduling practice WorkKeys tests could help students 
improve their scores. Even taking WorkKeys once for 
practice would likely help students understand which 
areas they need to study and, at the very least, teach 
students how to better manage their time when taking the 
assessment. Integrating ACT® WorkKeys® National Career 
Readiness Certificate® (NCRC®) levels into school honor 
rolls would likely increase test motivation. These simple 
actions and others outlined in this report can help high 
schools develop more career-ready high school students. 
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Introduction 


Many career and technical education (CTE) students in Alabama high schools take 

the ACT® WorkKeys® Applied Math, Workplace Documents, and Graphic Literacy 
Assessments to measure their foundational workplace skills. Moreover, students can 
earn an ACT® WorkKeys® National Career Readiness Certificate® (NCRC®), which 
certifies skills to potential employers and postsecondary training programs (ACT, 2014). 
Consequently, the Alabama State Department of Education (ALSDE) has partnered 
with ACT Research to share data and explore relationships between educational 
programs and the foundational workplace skills measured by WorkKeys. 


This report focuses on Alabama high schools with greater-than-expected WorkKeys 
performance. That is, compared to other schools enrolling students with similar 
demographics and scores on the ACT® test, these schools exhibited higher average 
WorkKeys scores. Ten such schools were contacted to find out how they promote 
WorkKeys performance and career readiness more generally. The hope is that these 
schools can provide insights to other high schools in Alabama on increasing their 
WorkKeys scores. 


Method 


Statistical analyses estimated average WorkKeys performance, relative to 
expectations, for 351 high schools in Alabama (see Appendix for additional details). 
The result was not a simple ranking of schools in terms of WorkKeys performance. 
Rather, these analyses identified schools whose WorkKeys performance was 
particularly good compared to other schools with students having similar levels of 
academic achievement and similar demographics (e.g., gender, ethnicity). The ALSDE 
contacted 10 schools with greater-than-expected WorkKeys performance to ask the 
following questions: 


1. Does your school have any special programs to help get students career ready? 
2. How do your students prepare to take WorkKeys? 


3. How do you communicate the value of taking WorkKeys and earning a WorkKeys 
NCRC? 


4. Is WorkKeys recognized by local employers or postsecondary training programs? 
An administrator, teacher, or counselor from six of the 10 schools (60% response rate) 
responded to the request and consented to be named in this report. The following 
Alabama high schools are represented: 

¢ Booker T. Washington High School 

e Clarke County High School 


¢ Douglas High School 


ACT Research & Policy | Research Report | November 2020 4 


¢ Phil Campbell High School 
e Ramsay High School 


« Thomasville High School 


Results 


Career Readiness Programs 


In terms of career readiness programming, the surveyed schools most often reported 
providing flexible initiatives wherein students can go beyond standard high school 
coursework to prepare for careers. Several schools mentioned co-op programs where 
students work part-time while still taking classes. Some schools also mentioned dual- 
credit opportunities at local universities and community colleges. Finally, two schools 
mentioned coursework through which students earn work-based certifications. 


Another program, mentioned by two schools, was the Kuder Navigator program, which 
is an online education and career planning system. Several other career programs were 
mentioned at least once and are as follows: 


* Smart Work training program 

e Ready to Work program 

¢ Jobs for Alabama Graduates (JAG) 

¢ Future Teachers of Alabama (FTA) 

¢ Future Business Leaders of Alabama (FBLA) 

¢ Family, Career, and Community Leaders of Alabama (FCCLA) 


The majority of respondents also mentioned conducting career fairs. The mention of 
career fairs cut across several of the survey questions, as career fairs serve multiple 
purposes related to career readiness (e.g., networking, communicating the value of 
WorkKeys, encouraging test-taking motivation, understanding how local employers do 
(or do not) use WorkKeys for hiring). Lastly, Douglas High School mentioned providing 
additional career services for students with disabilities, such as mock interviews, job 
shadowing, and internships. 


Preparing for WorkKeys 


Student performance on the WorkKeys Applied Math, Workplace Documents, and 
Graphic Literacy Assessments is reported using level scores that correspond to specific 
knowledge and skills required in a broad range of occupations. ACT® WorkKeys® 
Curriculum provides a placement exam, instructional modules, practice exercises, and 
quizzes to evaluate mastery of Applied Math, Workplace Documents, and Graphic 
Literacy at Levels 3, 4, 5, 6, and 7. Students who score at a level of 3 or higher on all 
three tests earn a Bronze NCRC.' 
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All the survey respondents mentioned using formal training materials such as WorkKeys 
Curriculum to help students prepare to take WorkKeys. ALSDE has set up an account 
with ACT where online WorkKeys Curriculum materials are available for any public 

high school in Alabama. Prior research indicates that even brief engagement with 
WorkKeys Curriculum is associated with higher WorkKeys scores (Steedle & Hawk, 
2020). Additionally, all respondents dedicate regular class time to preparing students 

for the WorkKeys assessment. Several of the high schools use multiple class periods 
each week exclusively for WorkKeys test preparation. Two of the respondents administer 
multiple practice tests throughout the school year. Booker T. Washington High School 
uses the practice tests diagnostically to place students in study groups for customized 
instruction. Similarly, Thomasville High School uses ACT scores to place students into 
study groups. 


The Value of WorkKeys 


Students exhibit higher motivation on achievement tests when they understand 
the importance or value of an assessment. Schools reported many ways that they 
communicate the value of WorkKeys to their students. Here are a few examples: 


¢ Booker T. Washington High School invites back college students and employed 
graduates to share their experiences and discuss the value of WorkKeys testing. 
Booker T. Washington also categorizes their honor roll using NCRC distinctions (i.e., 
Bronze, Silver, Gold, and Platinum). 


¢ Clarke County High School provides students with a link to the ACT® Work Ready 
Communities website (https:/Awww.workreadycommunities.org/), which lists local 
employers that recognize WorkKeys. 


¢ Asan incentive, Douglas High School offers exemption from all semester exams for 
any one semester for any student who passes all parts of WorkKeys (i.e., Applied 
Math, Graphic Literacy, and Workplace Documents). 


¢ Teachers at Phil Campbell High School inform students about how employers use 
the WorkKeys NCRC to evaluate applicants who come directly from high school and 
those from college. 


¢« Ramsay High School provides graduating seniors with certificates and honors 
recognition for their performance on WorkKeys, which is similar to Booker T. 
Washington. 


¢ Thomasville High School communicates to students that over 350 of Alabama’s 
employers use the WorkKeys NCRC as part of their hiring process. 


WorkKeys Recognition 


All survey respondents said that WorkKeys is recognized by local employers. Clarke 
County High School also mentioned that their local community college uses WorkKeys 
test scores for all of its technical programs. There were several other anecdotes from 
survey respondents that warrant mention: 
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* Booker T. Washington High School said that many students return after graduation 
to request a copy of their scores because local employers are offering jobs for 
applicants with certain scores. 


¢ Clarke County High School mentioned that the Southwest Alabama Workforce 
Development Council is working to get all counties in the region recognized as ACT 
Work Ready Communities (https://www.workreadycommunities.org/). 


e Phil Campbell High School works closely with local businesses to make students 
aware of qualifications required to gain employment. 


Additional Resources 


To learn more ways that high schools promote career readiness, please refer to a prior 
research study conducted with data from Arkansas high schools (Steedle & Girdler, 
2020). High schools in Alabama, in Arkansas, and across the nation should have 
opportunities to learn from each other in pursuit of their shared goal of helping students 
achieve educational and workplace success. 


Appendix: Statistical Analysis 


Data 


WorkKeys data for this study were extracted from the ACT WorkKeys examinee 
database, which includes records from Alabama high schools that tested during the 
2018-2019 academic year. The WorkKeys examinee data were merged with ACT 
examinee data from spring 2018 statewide testing using name, date of birth, and 
zip code. Information about Alabama high school characteristics was gathered from 
publicly available data sets from the ALSDE website. 


Analysis 


High schools were included in analyses if they had 20 or more students with complete 
records (ACT scores, WorkKeys scores, ethnicity, and gender). This resulted in a 
sample of 44,337 students from 351 high schools. Descriptive statistics for students 
and schools were calculated for demographics and assessment scores. These data 
were used to fit a series of hierarchical linear models (with students nested in schools). 
Model fitting was repeated for each of the following combinations of assessments: ACT 
mathematics as a predictor of WorkKeys Applied Math, ACT reading as a predictor 

of WorkKeys Workplace Documents, and ACT science as a predictor of WorkKeys 
Graphic Literacy. Model 1 (shown below) is the unconditional model with no predictor 
variables. Model 2 adds student-level predictors (ACT score, non-White indicator, 
female indicator), and Model 3 adds school-level predictors (mean ACT, percentage 
non-White, percentage female, and percentage eligible for free or reduced-price 
lunch). Note that percentage eligible for free or reduced-price lunch was only known 

at the school level (not for individual students). Model 4 includes the predictors with 
consistently significant coefficients in Models 2 and 3 (for at least two of the three 
analyses). Finally, the school-level residuals were estimated based on Model 4 to 
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indicate WorkKeys performance relative to expected. The residuals were transformed 
to standard deviation units, and the school-level residuals from all three analyses were 
merged into a single data set. The average residual was calculated for each school, 
and the data were sorted to identify the schools with the highest average residuals. 


Model 1 
WKij = Boj + Mj 


Boj = Yoo + Uoj 
Model 2 
WK;jj = Boj + Bij ACT; + B2;NonWhite;; + B3;Female;; + 7; 
Boj = Yoo + Uoj 
Bij = Y10 
Bo; = Y20 
B3; = Y30 
Model 3 
WK;j; = Boj + B1jACT;; + B2jNonWhite;; + 63;Female;; + 7; 


Bo; = Yoo + Yo1 ACT; + Yo2Pct_NonWhite; + Vo2Pct_Female; + Yo2Pct_FRPL; + ug; 
Jj j j Jj j J 


Bij = VY10 
B2; = Y20 
3; = Y30 


Model 4 
WKij = Boj + Bij ACT;; + B2j;NonWhite;; + B3;Female;; + 7; 
Boj = Yoo + Yo1 ACT; + Uo; 
Bij = 10 
B2j = ¥20 


B3; = Y30 
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Results 


Student Descriptive Statistics 


The sample was 50.8% female, 30.7% Black, 55.4% White, and 6.2% Hispanic/Latino 
(Table 1). As shown in Table 2, mean ACT scores ranged from 17.8 (mathematics) to 
18.7 (reading). The mean WorkKeys scale scores were 78.5 for Applied Math, 79.8 

for Workplace Documents, and 79.3 for Graphic Literacy. WorkKeys performance is 
also reported by level scores ranging from 3 to 7. The mean WorkKeys scale scores 
corresponded to level scores of 4, 4, and 5, respectively. Note that scale scores (rather 
than level scores) were used in most analyses because they provide more fine-grained 
indicators of student achievement. By attaining certain level scores, students can earn 
a WorkKeys NCRC at the Bronze, Silver, Gold, or Platinum level. Of the Alabama high 
school students, 23.5% earned Bronze, 38.4% earned Silver, 17.2% earned Gold, and 
12.5% earned Platinum (Table 1). 


The distributions of ACT and WorkKeys scores are shown in the diagonal elements of 
Figure 1. The off-diagonal elements show scatter plots of ACT scores and WorkKeys 
scores and corresponding correlations, all of which were statistically significant (p 

< .001). The correlations between ACT scores (English, mathematics, reading, and 
science) and WorkKeys scores ranged from 0.61 to 0.76. The correlation between ACT 
mathematics and Applied Math was the strongest. 


Table 1. Student Level Distributions of Gender, Ethnicity, and NCRC Level 


Variable Level N Percentage 
Gonder Female 22,010 50.8% 
Male 21,027 49.2% 
Black/African American 13,591 30.7% 
American Indian/Alaska 354 0.8% 
Native 
White 24,547 55.4% 
Ethnicity Hispanic/Latino 2,765 6.2% 
Asian 620 1.4% 
Native Hawaiian/Other 51 0.1% 
Pacific Islander 
Two or more races 1,818 4.1% 
PNR/Missing rose 1.3% 
Non-Qual. 3,697 8.3% 
Bronze 10,424 23 O/o 
NCRC level Silver 17,031 38.4% 
Gold 7,645 17.2% 
Platinum 5,540 12.5% 


Total sample size 44,337 
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Table 2. Student Level ACT and WorkKeys Descriptive Statistics 


Variable Mean S)B) 25th Percentile Median 75th Percentile 
ACT English 18.0 Gr 18 17 22 
ACT mathematics 17.8 4.4 15 16 20 
ACT reading 18.7 5.9 14 18 22 
ACT science 18.4 5.0 15 18 22 
ACT Composite 18.3 4.9 15 AT 21 
Applied Math 78.5 4.6 75 78 82 
Workplace Documents 79.8 3.9 Vi 80 82 
Graphic Literacy 79.3 4.0 a 79 82 


Figure 1. Correlation Matrix of ACT and WorkKeys Assessment Scores 
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School Descriptive Statistics 


Table 3 provides descriptive statistics for the distributions of demographic variables 
across schools. These variables include sample size, percentage female, percentage 
ethnicity, percentage eligible for free or reduced-price lunch (FRPL), percentage of 
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teachers with a master’s degree, the overall college and career readiness (CCR) 
percentage, the percentage demonstrating CCR on the ACT, the percentage 
demonstrating CCR on WorkKeys, and the Alabama graduation rate. Table 4 
summarizes the distributions of mean ACT and WorkKeys scores across the 351 
Alabama high schools in the sample. 


Table 3. School Level Characteristic Distributions 


25th 75th 
Variable Mean SD Percentile Median Percentile 
Sample size 126.3 (OP Te 54 93 160.5 
Female 50.6% 6.1% 47.2% 50.4% 53.9% 
Native American 0.9% 2.1% 0.0% 0.0% 1.0% 
Asian 0.8% 2.1% 0.0% 0.0% 1.0% 
Black/African American 29.4% 31.5% 3.8% 18.2% 46.2% 
Hispanic/Latino 5.9% 7.0% 2.1% 4.1% 7.2% 
Pacific Islander 0.1% 0.2% 0.0% 0.0% 0.0% 
Two or more races 4.0% 3.0% 2.0% 3.6% 5.3% 
White 57.7% 30.3% 37.9% 66.7% 81.8% 
FRPL 51.7% 17.0% 41.2% 52.9% 62.8% 
Master’s degree Sa 76 6.5% S226 55.8% 58.9% 
Overall CCR 72.4% 14.8% 65.3% 74.7% 82.4% 
ACT CCR 48.6% 16.6% 38.2% 49.1% 59.4% 
WorkKeys CCR 55.7% 15.6% 48.0% 57.2% 66.0% 
Alabama graduation rate 89.4% 5.0% 86.9% 90.3% 93.3% 


Table 4. School Level ACT and WorkKeys Descriptive Statistics 


25th 75th 
Variable WV KexeTal S)B) Percentile Median Percentile 
ACT English leo 2.4 16.8 NZ 18.6 
ACT mathematics 172 Wee 16.2 Af2 18.0 
ACT reading 18.1 Do 17.0 18.1 19.2 
ACT science 17.8 1.9 16.6 17.7 18.9 
ACT Composite N00 2.0 16.6 Nol 183.7 
Applied Math 78.1 1.8 (12 78.4 79.3 


Workplace Documents 79.5 1.3 78.8 79.6 80.4 
Graphic Literacy 79.1 1.4 18:2 79.2 80.0 
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Hierarchical Linear Modeling 


A series of four hierarchical linear models were fit to the data to evaluate various 
predictors of WorkKeys scores. Model 1 is the unconditional model, which includes no 
predictor variables. Fitting the unconditional model provided estimates of the between- 
school variance (7,,) and pooled within-school variance (07). These values indicated 
that between 10.8% and 14.8% of the variance in WorkKeys scores was between 
schools (the intra-class correlation). Model 2 included several student-level predictors 
of WorkKeys scores: ACT score, non-White indicator, and female indicator. Depending 
on the subject area, the inclusion of these variables accounted for 78.1—86.6% of the 
between-school variance (see reduction in 7,,) and 41.1-53.0% of the pooled within- 
school variance (see reduction in 07). The coefficients for ACT score, non-White, and 
female were each statistically significant in Model 2 for all three assessments. 


Model 3 added several school-level predictors: mean ACT score, percentage of non- 
White students, percentage of female students, and percentage of students eligible 
for free or reduced-price lunch (based on ALSDE statistics, not the sample). For each 
subject area, the inclusion of these variables accounted for a very small percentage 

of the remaining between-school variance. Of those predictors, only mean ACT was 
statistically significant for two of the three assessments, so all others were omitted in 
Model 4. Model 4 included ACT score, non-White indicator, and female indicator as 
student-level predictors and mean ACT score as a school-level predictor. To some 
extent, the non-White indicator variable likely served as a proxy for other (unmeasured) 
variables correlated with ethnicity (e.g., socioeconomic status, educational opportunity, 
school resources, etc.). Model 4 was treated as the final model. Model parameter 
estimates allowed for estimation of the school-level residual (u,,), which indicated 
average WorkKeys performance relative to expected. Each school had three residuals 
(one for each WorkKeys assessment). Those residuals were averaged to identify 
schools with greater-than-expected WorkKeys scores. 
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Table 5. HLM Parameter Estimates 


[OU] Kero) gare) 


Applied 
Math 


Workplace 
Documents 


SW Zan] eye) 


Description 
Intercept 
Student-level ACT 
mathematics 
Student-level non- 
White 
Student level female 
School-level mean 
ACT mathematics 


School-level % non- 
White 


School-level % 
female 

School-level % FRPL 
School intercept 
variance 

Student residual 
variance 

Intercept 


Student-level ACT 
reading 


Student-level non- 
White 

Student level female 
School-level mean 
ACT reading 


School-level % non- 
White 


School-level % 
female 

School-level % FRPL 
School intercept 
variance 


Student residual 
variance 


Model 1 Model 2 
"8150 900 Dds = 
O:yse 
=1.094*** 
=0.185"" 
SS) 0.417 
1711 8.414 
19.005" 7.983" 
0.420*** 
-0.485*** 
0.370""* 
1.630 0.349 
13.472 7.931 


Model 3 
64.619*** 


0.762""* 


S021 


=0.185""" 


0.053 


=0.0710""" 


0.005 


0.004 


0.310 


8.411 


68.696*** 


0.419*** 


-0.459*** 


0.376*** 


O58" 


0.001 


0.001 


0.006* 


0.296 


1229 


Model 4 
64.053*** 


Cfo 


=1,083"°* 
=0.156°"" 


C1091 


0.401 


8.413 


70.096*** 


0.418*** 


-0.460*** 


0.372*** 


021057 = 


0.305 


7.930 


ACT Research & Policy | Research Report | November 2020 


Table 5. HLM Parameter Estimates—continued 


(@lU Kero) ants, 


Graphic 
Literacy 


Notes 


SW Zan lele)| 


Description NV Core (=\ima NV Coo (=1072 
Intercept MOVs BI.Sis 
Student-level ACT 0.520*** 
science 
Student-level non- one 
White =Oio22 
Student level female 0.216*** 


School-level mean 
ACT science 


School-level % non- 
White 


School-level % 
female 


School-level % FRPL 


School intercept 


‘ 1.910 0.419 
variance 
Student residual 13.950 7.933 
variance 


VCore (=) lee) 
68.273*** 


0.5217" 


06657 


0.216"** 


0.082* 


-0.003 


0.002 
0.003 


0.397 


1.922 


1. Silver NCRC = score of 4 or higher on all three tests; Gold NCRC = score of 5 or 
higher on all three tests; Platinum NCRC = score of 6 or higher on all three tests. 
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